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Abstract — Recently, lattice reduction (LR) technique has 
caught great attention for multi-input multi-output (MIMO) 
receiver because of its low complexity and high performance. 
However, when the number of antennas is large, LR-aided linear 
detectors and successive interference cancellation (SIC) detectors 
still exhibit considerable performance gap to the optimal max- 
imum likelihood detector (MLD). To enhance the performance 
of the LR-aided detectors, the LR-aided K-best algorithm was 
developed at the cost of the extra complexity on the order 
0{NtK + NtK^), where A^t is the number of transmit antennas 
and K is the number of candidates. In this paper, we develop an 
LR-aided K-best algorithm with lower complexity by exploiting 
a priority queue. With the aid of the priority queue, our analysis 
shows that the complexity of the LR-aided K-best algorithm 
can be further reduced to 0{NtK + NtKlog^^K)). The low 
complexity of the proposed LR-aided K-best algorithm allows us 
to perform the algorithm for large MIMO systems (e.g., 50x50 
MIMO systems) with large candidate sizes. Simulations show 
that as the number of antennas increases, the error performance 
approaches that of AWGN channel. 

Index Terms — Lattice reduction, native lattice detection, MIMO, 
K-best algorithm 

I. Introduction 

Designing reliable and computationally efficient multi-input 
multi-output (MIMO) detectors has been a longstanding chal- 
lenge for wireless MIMO communications. Despite its optimal 
error performance, the maximum likelihood detector requires 
considerably high complexity, especially when the number of 
antennas is large [IJ. In contrast, linear detectors (LDs) and 
successive interference cancellation (SIC) detectors require 
polynomial complexity but suffer from significantly degraded 
error performance. Recently, to improve the error performance 
of LDs and SIC detectors, lattice reduction (LR)-aided detec- 
tion is proposed IJ], 10, iH, which show that LR-aided LDs 
can achieve the same diversity as the MLD. 

Although significant performance improvement for LR- 
aided LDs and SIC detectors is found, the LR-aided detectors 
still exhibit considerable performance loss to the MLD. In 
addition, as the number of antennas increases, the gap between 
the LR-aided detectors and the MLD increases significantly 
Is). To further bridge the gap, the LR-aided K-best algorithm is 
proposed in |6|, |7|. Compared to the conventional K-best, the 
LR-aided K-best algorithm has no boundary information about 
the symbols in the lattice-reduced domain, i.e., the possible 
children for each layer can be infinite. To find the K best 



partial candidates from the infinite children set, the algorithm 
in [6J replaces the infinite set with a finite subset of the 
children. To reduce the complexity of generating the subset, 
study in (7] develops an on-demand child expansion based on 
the Schnorr-Euchner (SE) strategy. 

In this paper, we further reduce the complexity of finding the 
K best partial candidates from the infinite children set for each 
layer by exploiting the on-demand child expansion and a prior- 
ity queue. We find that, with the aid of the priority queue, the 
complexity can be reduced from O^NfK + NtK"^) in f6l, fTl 
to 0{N'^K + NtK\og2{K)). The lower complexity in terms 
of the candidate sizes K allows us to perform MIMO with 
large candidate sizes, resulting higher error performance . Our 
simulations show that, combined with the minimum-mean- 
square-error (MMSE) regularization, the proposed LR-aided 
K-best algorithm can achieve near-optimal error performance 
for large MIMO systems with large constellation sizes (e.g., 
50x50 MIMO with 256QAM). 

The rest of the paper is organized as follows. Section |ll] 
introduces the system model and LR-aided detectors. Section 
Hn] discusses the proposed LR-aided K-best algorithm. Section 
|IV]shows the numerical results. Section |V]concludes the paper. 

Notation: Superscript ^ denotes the transpose. The real and 
imaginary parts of a complex number are denoted as fH[] and 
3[]. Upper- and lower-case boldface letters indicate matrices 
and column vectors, respectively. Ai^k indicates the (i, fc)th 
entry of matrix A. Ijv denotes the N x N identity matrix, 
Onxl is the N x L matrix with all entries zero, and Inxl is 
the NxL matrix with all entries one. Z is the integer set. E{ ■ } 
denotes the statistical expectation. || • || denotes the 2-norm. 

II. System Model 

The transmission model of a V-BLAST MIMO system with 
Nf transmit antennas and Nr receive antennas is 



= HV 



(1) 



where s"^ — 



'l7 ^2: 



s^J^, {s'l e S'^) is the complex 
information symbol vector with S'^ being a constellation set of 
QAM, H*^ is an NrX Nt, {Nr > Nt) complex channel matrix, 
whose entries are modeled as independent and identically 
distributed (i.i.d.) complex Gaussian variables with zero means 
and unit variances, y^ = [2/1,2/21' ■■ ^Un]'^ i^ th^ received 
signal vector, and w^ = [w^, Wj, • • • , w'j^ ]^ is the complex 



additive white Gaussian noise (AWGN) vector with zero mean 
and covariance NqIn^. 

Given the complex signal model in ([T]i, the equivalent real 
signal model is 



3[y1 



Hs + w. 



3M 






(2) 



where s = [si, S2, • ' ' i s-zNt]"^ with Si ^ S and S is the con- 
stellation set of PAM as {-y/M+l, -\/M+3, • • • , VM-1}. 

A. LR-aided Detectors 

Given the model in Eq. (|2]l, the MLD is 

-ML 



arff inin 



Hs||^ 



(3) 



which is generally non-deterministic polynomial hard (NP- 
hard). To reduce the high complexity of the MLD, the LR- 
aided detection is proposed. 

Since the LR-aided detection only works for infinite lattice, 
we first remove the boundary constraints in Eq. (|3) and obtain 
the following relaxed problem 

2 



arg^ 



mm 



Hill 
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where U is the unconstrained constellation 
{• • • , — 3, — 1, 1,3, • • •}. Since s may not be 
QAM symbol, a quantization step can be applied 



set as 
a valid 



gNLD 



Q(§), 



(5) 



where Q(-) the symbol- wise quantizer to the constellation set 
S. 

In essence, the unconstrained detection in Eq. ^ is the 
naive lattice detection (NLD) studied in |8|, |9|. The closest 
point search algorithm lITOl (e.g., sphere decoding algorithm) 
can find the optimal solution to (|4|i. However, one issue of the 
NLD is that it is not diversity-multiplexing tradeoff (DMT) 
optimal in general |J9l, i.e., the NLD is suboptimal in terms 
of diversity. To achieve the DMT optimality, the regularized 
lattice decoding is proposed as 1 8 1 
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where we adopt the MMSE regularization, i?{ss^} = cr^I, 
and H and y are the MMSE-extended matrix and the extended 
received signal vector as 



H 




2Nt 



y 

02Nt X 1 



(7) 



We call the detection in dSll the "NLD with MMSE." Unless 
stated otherwise, in the following, we focus on solving the 
NLD with MMSE in Eq. Q, but the discussed LR-aided K- 
best algorithm is also applicable to the NLD in Eq. (|4]i. 

To solve the NLD with MMSE in Eq. (O with lower 
complexity, the LR-aided detection performs LR on the matrix 



H to obtain a more "orthogonal" matrix H = HT, where T is 
a unimodular matrix, such that all the entries of T are integers, 
and the determinant of T is ±1. Given H and T, the NLD 
with MMSE becomes 



s = 2Targ min ||v 



Hz|| +l2ArtXl, 



(8) 



where y is the received signal vector after shifting and scaling 
as (y — Hl2Arjxi)/2 and s = 2Tz + l2Aftxi- Since H is 
more "orthogonal," the closest point search algorithm based 
on H can enjoy much lower complexity compared to that 
based on H in Eq. (|7|l ifTOl . However, since the problem in 
Eq. ^ is NP-hard, the complexity of the closest point search 
is still considerably high when Nt is large. To achieve low- 
complexity detection, the LR-aided MMSE-SIC detector finds 
a sub-optimal solution to (O with degraded error performance. 

III. LR-AIDED K-Best Algorithm 

To enhance the performance of the LR-aided MMSE-SIC 
detector, the LR-aided K-best algorithm IS, Q is proposed 
to find a "better" sub-optimal solution to (HJ. 

The LR-aided K-best algorithm first performs QR decom- 
position on H = QR, where Q is a 2{Nr + Nt) x 2Nt 
orthonormal matrix and R is a 2Nt x 2Nt upper triangular 
matrix. Then, the problem in dHJ is reformulated as 



s = 2T arg min 

zez2"t 



R2| 



L27VtXl- 



(9) 



where y = Q^y. 

Next, the LR-aided K-best algorithm performs the breadth- 
first search from the 2iVtth layer to the 1st layer For each layer 
(e.g., the nth layer), the algorithm computes the K best partial 



candidates fzi 



52 ■ • ■ ■ iZ]^' ], i.e., the K partial candidates 
with the minimum costs among all the children of the K 



partial candidates [z\ 



(n+l) (n+1) 



,("+l)l 



(n) 
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in the previous 



(n + l)st layer, where a partial candidate z^ ' in the nth layer 
and the cost associated with the partial 



(n) 

candidate is 
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'■i,2Nti 
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Input: Channel matrix H, received signal vector y. candidate size K 
Output: Hard-output estimate s 
H) Obtain H and y in Eq. Q 

(2) [H,T] = LR(H) 

(3) [Q, R] = QR(H) 

(4) y = QT(y-Hl2iVtxi)/2 



(5) 



(2iVt + l) 



"1 



[] , cost^ 



(2Nt+l) 



■■ 0, len = 1 



: 2 Aft downto 1 



[{zi")}f^„{cos4"'}f^,] 



(6) For n ■■ 

(7) _^ _^^ 

Find_Kbest_Children({z^"+^^ }|5Z?i , {cos^""^^^ Ijfi^i ) 

(8) len = K 

(9) End for 

(10) Output s = arg min. n,„^ d) ., J|y-Hsfc||2 

TABLE I 

A GENERAL DESCRIPTION OF THE LR-AIDED K-BEST ALGORITHM. 



Input: len paitial candidates of the (n + l)st layer {z~" IfelTi ^'"^ ^^^^ costs {costj^ Vk^i 
Output: K partial candidates of the nth layer {z\" }^_-i with their costs {cost^"''}^_-^ 



Line no. Description 



Complexity 



(1) 


For i = 1 to len 




(2) 


n — Vn l^j^n + 1 "-n.J^iJ 


0{Nt) 


(3) 


Zi = \ri/R„^n\ 


C(l) 


(4) 


Child, = [z„{zl"+'^)T]T 


0(1) or 0{Nt) 


(5) 


childcOSU = COs4" + ^^ + in - R«,nZ,)^ 


0(1) 


(6) 


stepi = sgn{ri/R„,n - Zi) 


0(1) 


(7) 


End for 




(8) 


Initialize a priority queue q with {childcosti}\'!J^-^ as the keys 


0(K) 


(9) 


For A: = 1 to i^ 




(10) 


Find the index i associated with the minimum key in q 


o(i) 


(11) 


z^" = childi 


0(i) or ©(Aft) 


(12) 


cosv-^ = childcosti 


C(l) 


(13) 


Zi = Zi + stepi 


C(l) 


(14) 


child, = [z,,{zl"+''>)^]'r 


C(l) 


(15) 


childcosti = COSt'f''^'^^ + {n - R„,nZif 


0(1) 


(16) 


stepi = —stepi — sgn(stepi) 


0(1) 


(17) 


Update q using childcosti as the new key 


0(log2(X)) 


(18) 


End for 




(19) 


Output {z(")}i^,,{co4"'}f^, 





TABLE II 

The proposed Find_Kbest_Children() SUBROUTINE FOR THE LR-AIDED K-BEST ALGORITHM. 



(n) 

We call a partial candidate of the nth layer zr- a child of a 
partial candidate of the (n + l)st layer z^" ' if and only if 



,(") 



[4:;^,(-r^o 



(n+l)sTlT An) 



e Z holds. 



A general description of the LR-aided K-best algorithm is 
given in Table U Note that only one partial candidate is in the 



{2Nt + l)st layer, where ■z\ 



{2Nt + l) 



represents the root node. 



From TableU the key task of the LR-aided K-best algorithm 
is how to efficiently find the K best partial candidates of 
each nth layer from all the children of the partial candidates 
of the previous [n + l)st layer. However, different from 
the K-best algorithm in s-domain, in which the number of 
children is finite due to the bounded constellation set S, each 
partial candidate in the LR-aided K-best algorithm has infinite 
possible children because no information about the boundary 
of z is available. 

To address the infinite children issue, the algorithm in |6| 
approximates the infinite children set with a finite set with 
NK children by expanding only N best children for each 
partial candidate of the (n + l)st layer. The algorithm then 
chooses the top K partial candidates for the nth layer from 
the NK children. Note that, when N = K, the K'^ children 
themselves contain at least K best partial candidates among all 
the children of the (n+l)st layer, and thus, the algorithm in [6l 
becomes an exact one. To further reduce the number of node 
expansions, the algorithm in Q employs the Schnorr-Euchner 
(SE) strategy to perform an on-demand child expansion, where 
a child is expanded if only if all its better siblings are expanded 
and chosen as the partial candidates of the nth layer. Although 
significant reduction on the node expansions is achieved in 
Q, both algorithms in E, Q require at least 0{NtK + K'^) 
operations to find the exact K best partial candidates for each 
layer 



In this paper, we further reduce the complexity of finding 
the exact K best partial candidates for each layer to 0{NtK + 
K log2{K)) by exploiting the on-demand child expansion and 
a priority queue. The pseudo code of the method is given 
in Table HIl Compared to the algorithm in [7|, the proposed 
algorithm employs a priority queue instead of a brute-force 
method in |7| to find a child with the minimum cost in line 
10. The priority queue can be implemented by a heap, which 
requires 0{1) operations to find the child with the minimum 
cost, 0{log2{K)) operations to maintain the heap if a key 
is changed (line 17), and 0{K) operations to initialize the 
heap with K elements (line 8). Thus, the overall complexity 
of the proposed method is 0{NtK + Klog2{K)), which is 
considerably lower than 0{NtK + K"^) in ||6l, Q when K 
is large. Note that, the complexity of lines 4 and 1 1 generally 
relies on the data structure of the implementation and is at 
most on the order 0{Nt) by using a memory copy method. 

IV. Numerical Results 

This section demonstrates the performance of the proposed 
LR-aided K-best algorithm for MIMO systems. For the LR 
algorithm, we adopt the LLL algorithm [11] with reduction 
quality parameter 5 = 0.99. The SNR is defined as the 
received information bit energy versus noise variance. 

Fig. [U displays the performance comparisons among the 
three K-best algorithms: /) the K-best algorithm in s-domain; 
//) the LLL-aided K-best algorithm for the NLD proposed in 
|7|; and ///) the LLL-aided K-best algorithm for the NLD with 
MMSE in Eq. dHJ, which is called the "LLL-aided MMSE K- 
best algorithm." From the figure, it is clear to see that, given a 
fixed number of K, both the LLL-aided K-best algorithm and 
the LLL-aided MMSE K-best algorithm achieve significant 
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Fig. 1. Performance comparisons of different K-best algorithms for 10x10 Fig. 3. Performance comparisons of the LLL-aided MMSE K-best algorithm 
MIMO systems with K = 5 and different QAMs. for a 32x32 MIMO system with 64QAM and different K. 
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Fig. 2. Performance comparisons of the LLL-aided MMSE K-best algorithm Fig. 4. Performance comparisons of the LLL-aided MMSE K-best algorithm 
for a 10x10 MIMO system with 64QAM and different K. for a 50x50 MIMO system with 256QAM and different K. 



gain over the K-best algorithm at high SNR, especially for 
high order QAMs. In addition, the LLL-aided MMSE K-best 
algorithm shows better performance than the LLL-aided K- 
best algorithm, especially for 4QAM, where the LLL-aided 
MMSE K-best algorithm has about 2.5 dB gain over the LLL- 
aided K-best algorithm at BER=10"'''. 

Fig. |2] depicts the performance comparisons among the 
LLL-aided MMSE K-best algorithm, the MLD, the complex 
MMSE-SIC, and the complex LLL-aided MMSE-SIC. We 
have the following observations: /) The LLL-aided MMSE-SIC 
obtains significant performance improvement over the MMSE- 
SIC, whose diversity is 1. //) The LLL-aided MMSE K-best 
algorithm further enhances the performance of the LLL-aided 
MMSE-SIC, where the LLL-aided MMSE K-best algorithm 
with K = 2 has more than 2 dB gain over the LLL-aided 
MMSE-SIC. ///) By increasing the number of candidates K, 
the performance of the LLL-aided MMSE K-best algorithm 
approaches that of the MLD, and when K = 15, the LLL- 



aided MMSE K-best can achieve almost the same performance 
as the MLD. 

Fig. [3] illustrates the performance of the different detectors 
for a 32x32 MIMO system with 64QAM. First, we observe 
that, compared to that in Fig. |2] the performance of the LLL- 
aided MMSE-SIC degrades significantly (e.g., the LLL-aided 
MMSE-SIC requires ai'ound 25.5 dB for 10x10 MIMO and 
ai-ound 29.3 dB for 32x32 MIMO at BER = IQ-^). Second, the 
LLL-aided MMSE K-best algorithm significantly improves the 
performance of the LLL-aided MMSE-SIC, which has about 5 
db loss to the LLL-aided MMSE K-best algorithm with K = 
5. Furthermore, the LLL-aided MMSE K-best algorithm with 
K = 1000 exhibits less than 2 dB loss to the unfaded AWGN 
bound at BER = 10"^. 

To further investigate the performance of the LLL-aided 
MMSE K-best algorithm for large MIMO systems with large 
constellation sizes, we compare the performance of the dif- 
ferent detectors for a 50x50 MIMO system with 256QAM in 



Fig. m From the figure, we find that the proposed LR-aided 
MMSE K-best algorithm with K = 128 and K = 4096 can 
achieve about 5 dB and 3 dB losses to the unfaded AWGN 
bound at BER = 10^^, respectively. 

V. Concluding Remarks 

In this paper, we proposed an LR-aided K-best algorithm by 
exploiting an on-demand child expansion strategy and a prior- 
ity queue. Our complexity analysis showed that the complexity 
of the proposed algorithm is 0{N^ K + NtK \og2{K)), which 
is lower than that of the exiting ones, especially for large can- 
didate size K. Our simulations demonstrated that, with large 
number of K, the proposed LR-aided K-best algorithm can 
achieve near-optimal performance for large MIMO systems. 
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